Robot control system, robot control method and user apparatus for robot control system

ABSTRACT

A robot control system, including: a mobile robot having a situation-detecting unit detecting surrounding situations; a user apparatus operating in a manner recognizable by a user based on signals detected by the situation-detecting unit; an action-detecting unit detecting a predetermined action of the user; a brain-activity-detecting unit detecting a brain activity of the user; and a robot-controlling unit controlling the robot. The user apparatus has a computing unit analyzing the brain activity of the user detected by the brain-activity-detecting unit to output a first-action instruction from the user to the robot. The robot controlling unit further controlling the robot in accordance with the first-action instruction output by the computing unit and controlling, when the predetermined action is detected by the action-detecting unit during operation of the user apparatus, the robot in accordance with a second-action instruction different from the first-action instruction output by the computing unit.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority fromJapanese Patent Application No. 2017-189798 filed on Sep. 29, 2017, thecontent of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION Field of the Invention

This invention relates to a robot control system, a robot control methodand a user apparatus for robot control system.

Description of the Related Art

Apparatuses for controlling actions of a remotely located robot inaccordance with instructions from a user are known. Such an apparatus isdescribed, for example, in Japanese Unexamined Patent Publication No.2006-289565 (JP2006-289565A). The apparatus taught by JP2006-289565Acontrols actions of a robot based on data representing user brainactivity acquired from a brain waveform sensor or the like.

A point of interest here is that a person who suddenly encounters anunexpected situation sometimes acts more appropriately reflexively thanconsciously by brain judgment. Therefore, in an apparatus like that ofJP2006-289565A arranged to control a robot based on data representinguser brain activity, the robot may not be able to act appropriately whenan unexpected situation is suddenly encountered.

SUMMARY OF THE INVENTION

An aspect of the present invention is a robot control system, including:a mobile robot having a situation detecting unit configured to detectsurrounding situations of the robot; a user apparatus configured tooperate in a manner recognizable by a user based on signals detected bythe situation detecting unit; an action detecting unit configured todetect a predetermined action of the user; a brain activity detectingunit configured to detect a brain activity of the user; and a robotcontrolling unit configured to control the robot, wherein the userapparatus has a computing unit configured to analyze the brain activityof the user detected by the brain activity detecting unit to output afirst action instruction from the user to the robot. The robotcontrolling unit further configured to control the robot in accordancewith the first action instruction output by the computing unit andconfigured to control, when the predetermined action is detected by theaction detecting unit during operation of the user apparatus, the robotin accordance with a second action instruction different from the firstaction instruction output by the computing unit.

Another aspect of the present invention is a robot control method forcontrolling a mobile robot having a situation detecting unit configuredto detect surrounding situations of the robot, including: detecting apredetermined action of a user; detecting and analyzing a brain activityof the user to output a first action instruction from the user to therobot; and controlling the robot in accordance with the first actioninstruction. The controlling the robot includes controlling the robot inaccordance with a second action instruction different from the firstaction instruction when the predetermined action is detected while auser apparatus operates in a manner recognizable by the user based onsignals detected by the situation detecting unit.

Further aspect of the present invention is a user apparatus for a robotcontrol system configured to operate in a manner recognizable by a usermanner based on signals sent from an external robot and indicatingsurrounding situations of the robot, including: a communicating unitconfigured to receive the signals indicating the surrounding situationsof the robot and configured to send a control signal to the robot; anaction detecting unit configured to detect a predetermined action of theuser; a brain activity detecting unit configured to detect a brainactivity of the user; and a computing unit configured to analyze thebrain activity of the user detected by the brain activity detecting unitto output a first action instruction from the user to the robot. Thecomputing unit further configured to output a second action instructiondifferent from the first action instruction output by the computing unitto the communicating unit as the control signal, when the predeterminedaction is detected by the action detecting unit.

BRIEF DESCRIPTION OF THE DRAWINGS

The objects, features, and advantages of the present invention willbecome clearer from the following description of embodiments in relationto the attached drawings, in which:

FIG. 1 is a diagram schematically illustrating overall configuration ofa robot control system according to an embodiment of the presentinvention;

FIG. 2 is a diagram showing a configuration of the user apparatus ofFIG. 1;

FIG. 3 is a block diagram showing a configuration of the user apparatusof FIG. 1 and peripheral apparatuses thereof;

FIG. 4 is a diagram showing an example of how action detectors of FIG. 3are worn;

FIG. 5 is a diagram showing structure of a robot of FIG. 1;

FIG. 6 is a block diagram showing configuration overview of a managementserver of FIG. 1; and

FIG. 7 is a flowchart showing an example of processing performed by acomputing unit of FIG. 6.

DETAILED DESCRIPTION OF THE INVENTION

An embodiment of the present invention is explained below with referenceto FIGS. 1 to 7. The robot control system according to this embodimentof the present invention analyzes user brain activity, activates actionsof a robot located at a place apart from the user, and offers the userservices in a perceptible mode based on signals detected by the robot.

Although the robot can act individually in response to instructions fromthe user, an example is explained in the following in which the robotacts not individually but also together with third persons such as theuser's family. The robot control system of the following example isadapted to deal with a situation in which a user arranges for a robot tojoin his or her family (or family member) in place of the user when theygo out by themselves (on a trip, for example) without the user whocannot join them for some reason and is particularly adapted to providethe user with information obtained by the robot in such a situation. Asa result, the user can, for example, enjoy a family outing just as iftraveling with family even while staying home.

FIG. 1 is a diagram schematically illustrating overall configuration ofa robot control system 100 according to an embodiment of the presentinvention. As shown in FIG. 1, the robot control system 100 isconfigured to comprise user apparatus 10 worn by a user 1 at point A, arobot 20 located at point B and operated by the user 1 at point A, and amanagement server 30.

The user apparatus 10, robot 20 and management server 30 arecommunicably interconnected by a network 2 including internet lines orother wireless communication network connection. Point A is, forexample, the home of the user 1 and point B is a place remote from pointA and can be located in another region. Point A and point B can even bein different countries.

The robot 20 at point B is rented at a shop 3 at or near point B. Inother words, the robot 20 is rented from the shop 3 visited by thefamily of the user 1 and participates in the user's family's activitiesat point B. The family of user 1 returns the robot 20 to the shop 3after finishing their trip activities at point B. Each robot 20available from the shop 3 has its own pre-assigned ID.

FIG. 2 is a diagram showing a configuration of the user apparatus 10. Asshown in FIG. 2, the user apparatus 10 is a wearable computer shapedoverall like a helmet, for example, and is worn on the user's head. Theuser apparatus 10 is provided with multiple sensors (brain activitydetectors 11) for detecting brain activity (brain activity signals) ofthe user including, inter alia, brain wave sensors for detecting userbrain waves or magnetoencephalographic waves, and brain blood flowstate. Namely, the user apparatus 10 is equipped with a so-calledbrain-machine interface (BMI) that detects user thoughts and intentionsfrom brain activity signals and achieves mechanical operations withoutusing body motions.

In addition, the user apparatus 10 has a display 12, microphone 13, andspeakers 14. The display 12 is, for example, a nontransparenthead-mounted display positioned to surround both of the user's eyes anddisplays camera images taken by the robot 20. The microphone 13 ismovably directed toward the user's mouth and receives voice signalsproduced by user utterances. The speakers 14 are located near the user'sears and produce voice output.

FIG. 3 is a block diagram showing the configuration of the userapparatus 10 and peripheral apparatuses thereof. As shown in FIG. 3, theaforesaid brain activity detectors 11, display 12, microphone 13 andspeakers 14 are wire or wirelessly connected to a controller 15. Awireless unit 16, input unit 17, and action detectors 18 areadditionally connected to the controller 15 by wired or wirelessconnection.

The brain activity detectors 11, display 12, microphone 13, speakers 14,controller 15, wireless unit 16 and input unit 17 are, for example,built into the user apparatus 10. The input unit 17 is configured tocomprise switches and a touch panel or the like operated by the user.Various data including user personal data and the like are input throughthe input unit 17.

The action detectors 18 are worn on the user's body. FIG. 4 is a diagramshowing an example of how the action detectors 18 are worn. In theexample of FIG. 4, the action detectors 18 are shown to be worn on thearms and legs of the user. The action detectors 18 can instead be wornat other parts of the body insofar as they are at least worn at partsthat move reflexively when the wearer suddenly encounters an unexpectedsituation, i.e., at parts that move unconsciously independently of brainjudgment. Suddenly arising situations that cause reflexive reactions areones like the sudden appearance of a thing or person before one's eyesor sudden emergence of something that makes one likely to stumble orfall. The action detectors 18 comprise, inter alia, acceleration sensorsfor detecting acceleration, muscle potential sensors for detectingmuscle electrical potential, and bend sensors for detecting joint angle.Presence/absence of reflex motion can be determined from signalsproduced by the action detectors 18.

As shown in FIG. 3, the controller 15 includes a microcomputercomprising a CPU or other computing unit 15A, a memory unit 15B equippedwith ROM, RAM and the like, and other peripheral circuits. Thecontroller 15 controls the wireless unit 16 and communicates with themanagement server 30. In response to instructions from the controller15, the management server 30 is sent robot 20 action instructions basedon signals from the brain activity detectors 11, microphone 13, actiondetectors 18, and so on. In addition, the controller 15 sends thedisplay 12 and speakers 14 control signals based on signals receivedfrom the management server 30.

FIG. 5 is a diagram showing structure of the robot 20. As shown in FIG.5, the robot 20 is a humanoid robot having a head, trunk, two arms andtwo legs, and is capable of autonomous movement by biped walking. Therobot 20 is nearly as tall as a human adult, with a height of around 140to 160 cm, for example.

The robot 20 is equipped with multiple sensors possessing detectioncapabilities corresponding to the “five senses” traditionally recognizedas sensing capabilities enabling humans to perceive the external world,namely, with at least one each of a vision sensor 211, sound sensor 212,touch sensor 213, smell sensor 214, and taste sensor 215. The sensors211 to 215 output signals corresponding to the human five senses (fivesense signals) as detection signals.

The vision sensor 211 comprises a camera having an imaging unitcomprising a CMOS sensor, CCD sensor or other image sensor located at aneye position of the robot 20 and a lens, a drive unit for driving theimaging unit vertically and laterally, and a zoom mechanism forenlarging and shrinking subject images. The vision sensors 211 takeimages (video images) around the robot 20. The sound sensor 212comprises a microphone located, for example, at an ear position of therobot 20. The sound sensors 212 pick up sounds around the robot 20. Thetouch sensor 213 comprises a force sensor located, for example, at ahand position of the robot 20. The touch sensors 213 detect externalforce acting on the hands of the robot 20. The smell sensor 214 islocated at a nose position of the robot 20 and detects smells. The tastesensor 215 is located at a mouth position of the robot 20 and detectstastes.

The robot 20 additionally comprises at least one each of an actuator 22,speaker 23, input unit 24, GPS sensor 25, controller 26, and wirelessunit 27. The actuators 22 include multiple motors or the like provided,for example, at associated joints of the robot 20. Robot 20 actions areimplemented by driving the actuators 22. The speaker 23 is located atthe mouth position of the robot 20 and produces voice output. The inputunit 24 is equipped with a power switch and various other switches. TheGPS sensor 25 receives GPS signals from GPS satellites. Position of therobot 20 can be detected using signals from the GPS sensor 25.

The controller 26 includes a microcomputer comprising a CPU, ROM, RAMand the like. The controller 26 controls the wireless unit 27 andcommunicates with the management server 30. For example, the controller26 sends the five sense signals output by the sensors 211 to 215 and thesignals from the GPS sensor 25 to the management server 30. In addition,the controller 26 sends control signals to the actuators 22, speaker 23and so on based on signals received from the management server 30.

FIG. 6 is a block diagram showing an example configuration overview ofthe management server 30. As shown in FIG. 6, the management server 30comprises an input unit 31, display unit 32, wireless unit 33 andcontroller 34. Optionally, the input unit 31 and display unit 32 can beomitted.

The controller 34 comprises a CPU or other computing unit 34A, a memoryunit 34B equipped with ROM, RAM, hard disk and the like, and otherperipheral circuits. The controller 34 controls the wireless unit 33 andcommunicates with the user apparatus 10 and the robot 20. The computingunit 34A comprises a robot controlling portion 341 and a user apparatuscontrolling portion 342 as functional constituents.

The robot controlling portion 341 generates actuation signals for theactuators 22 of the robot 20 based on robot 20 action instructionsreceived from the user apparatus 10, i.e., based on action instructionsresulting from user brain activity. These action signals are then sentthrough the wireless unit 33 to the robot 20. In response to theseactuation signals, the controller 26 of the robot 20 outputs controlsignals to the actuators 22. As a result, the robot 20 can be made toact on behalf of the user 1 in accordance with intention of the user 1.The robot controlling portion 341 can also use signals from themicrophone 13 of the user apparatus 10 to reproduce the user's voicethrough the speaker 23 of the robot 20.

The user apparatus controlling portion 342 generates action signals forthe user apparatus 10 based on signals (five sense signals) from thesensors 211 to 215 of the robot 20. These actions signals are thentransmitted through the wireless unit 33 to the user apparatus 10. Forexample, picture signals based on signals detected by the vision sensors211 are transmitted. In response to these picture signals, thecontroller 15 of the user apparatus 10 outputs control signals to thedisplay 12 to display 3D pictures obtained from the vision sensors 211on the display 12. The user apparatus controlling portion 342 can alsouse signals from the sound sensors 212 to reproduce voices through thespeakers 14 of the user apparatus 10.

In the so-configured robot control system 100, when the robot 20encounters a sudden unexpected situation in the course of acting inresponse to instructions from the user 1, the situation concerned ispicked up by the vision sensors 211 and displayed on the display 12. Ifat such time, the user 1 should produce a brain activity signal thatresults in the robot 20 being instructed to act to avoid the suddenincident (to perform an avoidance action), the action of the robot 20 inaccordance with this instruction would not be a robot 20 reflex actionand performance of an avoidance action by the robot 20 would be delayed.In the present embodiment, the robot control system 100 is configured asset out in the following in order to prevent such action delays by therobot 20.

As shown in FIG. 3, the computing unit 15A of the controller 15 of theuser apparatus 10 has as functional constituents an inputting portion151, a brain activity analyzing portion 152, an action determiningportion 153, and an outputting portion 154.

The inputting portion 151 receives signals from the sensors 211 to 215of the robot 20 through the wireless unit 16 and signals directly fromthe brain activity detectors 11, microphone 13, input unit 17 and theaction detectors 18.

The brain activity analyzing portion 152 uses signals of the brainactivity detectors 11 input to the inputting portion 151 to analyzebrain activity of the user 1 and specify action instructions of the user1 to the robot 20. The specified action instructions are sent throughthe wireless unit 16 to the management server 30, and actions of therobot 20 are controlled based thereon. Optionally, the wireless unit 16can be configured for not only wireless but also wired communicationconnection.

The action determining portion 153 determines whether actions of theuser 1 detected by the action detectors 18 are reflex actions. Since theuser 1 sees pictures acquired by the robot 20 displayed on the display12 in real time, the user 1 tends to identify with the robot 20.Therefore, when the robot 20 encounters a sudden unexpected situation,the user 1 reflexively reacts in response to the situation. For example,as indicated by double-dot chain lines in FIG. 4, the user 1 reflexivelyreacts in certain ways without any brain judgment on the part of theuser 1, such as by thrusting out arms or jerking legs forward. Theaction determining portion 153 uses signals from the action detectors 18to determine whether the user 1 had such a reflex action. Specifically,presence/absence of a reflex action is determined by determining whetherdetection value, change in detection value or detection value change perunit time of one or more action detectors 18, e.g. one or moreacceleration sensors and/or muscle potential sensors, reached orexceeded a predetermined value.

The outputting portion 154 outputs different action instructions to therobot 20 depending on the result of determination by the actiondetermining portion 153. Specifically, when the action determiningportion 153 determines that an action of the user 1 is not a reflexaction, the outputting portion 154 outputs an action instruction inaccordance with the user 1 brain activity determined by brain activityanalyzing portion 152 analysis (first action instruction). On the hand,when the action determining portion 153 determines that an action of theuser 1 is a reflex action, the outputting portion 154 outputs an urgentaction instruction stored in the memory unit 15B in advance (secondaction instruction). In either case, the output action instruction istransmitted through the wireless unit 16 to the management server 30,whereby the robot 20 is controlled in accordance with the transmittedaction instruction.

The second action instruction is an instruction for prompting the robot20 to act in response to a sudden unexpected situation. Examples ofsecond action instructions include, inter alia, an instruction toimmediately halt walking activity of the robot 20, an instruction tomove the robot 20 backwards, and an instruction to move the arms or legsof the robot 20 in a certain manner. When a reflex action of the user 1is detected, the outputting portion 154 outputs a second actioninstruction matched to detected behavior of the user 1 selected fromamong multiple second action instructions stored in the memory unit 15Bin advance. Optionally, a configuration can be adopted that responds todetection of a reflex action of the user 1 by outputting a certainpredetermined second action instruction regardless of how the user 1reacted.

FIG. 7 is a flowchart showing processing performed by the computing unit15A of the user apparatus 10 in accordance with a program stored in thememory unit 15B in advance, particularly processing related to output ofaction instructions to the robot 20. The processing represented by thisflowchart is, for example, commenced upon input to the robot 20 from theuser apparatus 10 of an instruction to activate the robot 20 and isrepeated at predetermined intervals.

First, in S1 (S: processing Step), signals are read from the brainactivity detectors 11 and the action detectors 18. Next, in S2, theaction determining portion 153 determines based on the signals from theaction detectors 18 whether action of the user 1 is reflex action. Whenthe result in S2 is NO, the program goes to S3, in which the brainactivity analyzing portion 152 analyzes brain activity of the user 1based on signals from the brain activity detectors 11 and specifies anaction instruction from the user 1 to the robot 20.

Next, in S4, the outputting portion 154 outputs a first actioninstruction based on the analysis result in S3, whereafter, in S5, theoutput first action instruction is sent to the management server 30. Asa result, the robot 20 acts in accordance with the brainactivity-derived instruction of the user 1. Pictures picked up by thevision sensors 211 of the robot 20 at this time are displayed on thedisplay 12.

On the other hand, when the action of the user 1 is determined in S2 tobe reflex action, the program goes to S6. In S6, the outputting portion154 selects from the memory unit 15B a second action instruction matchedto the reaction of the user 1 and outputs the selected second actioninstruction. Next, in S5, the output second action instruction is sentto the management server 30. As a result, the robot 20 acts withoutreceiving a user 1 brain-derived instruction. Since the robot 20 cantherefore instantaneously take appropriate action, just like a reflexaction, in response to the sudden unexpected situation, it can suitablyrespond to the sudden incident.

The present embodiment can achieve advantages and effects such as thefollowing:

(1) The robot control system 100 according to the present embodimentincludes: the mobile robot 20 having the various sensors 211 to 215 fordetecting surrounding circumstances; the user apparatus 10 that operatesin a manner perceivable by the user 1 based on signals detected by thesensors 211 to 215 (e.g., the vision sensors 211); the action detectors18 for detecting predetermined reflex actions of the user 1; the brainactivity detectors 11 for detecting brain activity of the user 1; thecomputing unit 15A (outputting portion 154) for analyzing brain activitydetected by the brain activity detectors 11 and outputting first actioninstructions from the user 1 to the robot 20; and the robot controllingportion 341 and controller 26 for controlling the robot 20 in accordancewith first action instruction output by the computing unit 15A (FIGS. 3and 5). When the user apparatus 10 is operating based on signals fromthe vision sensors 211, the robot controlling portion 341 etc. areresponsive to detection of a predetermined reflex action by the actiondetectors 18 for controlling the robot 20 in accordance with a secondaction instruction, distinct from a first action instruction, outputfrom the computing unit 15A (FIG. 7).

Thus when a predetermined reflex action of the user 1 is detected, therobot 20 is controlled in accordance with a second action instructiondistinct from a user 1 brain activity-derived first action instruction,so that it can always be made to act in an appropriate manner. In otherwords, the robot control system 100 adapted to control the robot 20using brain activity-derived instructions is further adapted to controlthe robot 20 using instructions not derived from brain activity when therobot 20 encounters a sudden unexpected situation. As a result, therobot 20 can be prompted to act in response to sudden unexpectedsituations in a suitable manner without delay.

(2) When a predetermined reflex action of the user 1 is detected by theaction detectors 18, the computing unit 15A selects a second actioninstruction matched to the detected action from the memory unit 15B,whereupon the robot controlling portion 341 etc. control the robot 20 inaccordance with the selected second action instruction. Since the secondaction instruction can therefore be decided in accordance with theunexpected situation suddenly encountered by the robot 20, the robot 20can be very suitably controlled. For example, the robot 20 can beprompted to perform arm and leg actions like those of the user 1.

(3) The robot 20 has the vision sensors 211 (cameras) for detecting itssurroundings, and the user apparatus 10 has the display 12 fordisplaying pictures picked up by the vision sensors 211 (FIGS. 2 and 5).The user 1 can therefore easily perceive circumstances around the robot20 in real time.

(4) The action detectors 18 can comprise sensors for detecting movementsof predetermined parts of the body of the user 1. In such case, therobot controlling portion 341 etc. respond to detection by the actiondetectors 18 of movement of a predetermined body part by a predeterminedamount (value) or greater by controlling the robot 20 in accordance witha second action instruction distinct from a first action instruction.Since reflex movements of the user 1 can therefore be accuratelydetected, the robot 20 can be appropriately controlled in accordancewith reflex movements.

(5) The action detectors 18 can be sensors that detect acceleration ofpredetermined parts of the user's body. In such case, the robotcontrolling portion 341 etc. respond to detection by the actiondetectors 18 of acceleration of a predetermined body part by apredetermined value or greater by controlling the robot 20 in accordancewith a second action instruction distinct from a first actioninstruction. Since this also enables accurate detection of reflexmovements of the user 1, the robot 20 can be appropriately controlled inaccordance with reflex movements.

(6) The robot control method according to an embodiment of the presentinvention, namely, the robot control method for controlling the mobilerobot 20 having the various sensors 211 to 215 for detecting surroundingcircumstances, comprises: detecting a predetermined action of the user1; detecting and analyzing brain activity of the user 1 and outputting afirst action instruction from the user 1 to the robot 20; andcontrolling the robot 20 in accordance with the output actioninstruction. Control of the robot 20 as termed here includes controllingthe robot 20 in accordance with a second action instruction distinctfrom a first action instruction when a predetermined reflex action isdetected while the user apparatus 10 is operating in a mannerperceivable by the user 1 based on signals detected by the sensors 211to 215. The robot 20 can therefore be prompted to act in an appropriatemanner without delay even when the robot 20 encounters some suddenunexpected situation. That is to say, the robot 20 can take appropriatereflex-like action.

(7) The user apparatus 10 for the robot control system according to theembodiment of the present invention operates in a manner perceivable tothe user 1 based on signals indicating circumstances around the robot 20transmitted from the remotely located robot 20. This user apparatus 10receives signals representing circumstances around the robot 20, andcomprises the wireless unit 16 as the communication unit fortransmitting control signals to the robot 20, the action detectors 18for detecting predetermined actions of the user 1, the brain activitydetectors 11 for detecting brain activity of the user 1, and thecomputing unit 15A (outputting portion 154) for analyzing brain activityof the user 1 detected by the brain activity detectors 11 and outputtingfirst action instructions to the robot 20 (FIG. 3). When the actiondetectors 18 detect a predetermined action, the computing unit 15Aoutputs to the wireless unit 16 as a control signal a second actioninstruction distinct from the first action instruction output by thecomputing unit 15A. Owing to the configuration of the user apparatus 10in this manner, the robot 20 can be prompted to take appropriate actionwith respect to sudden incidents without delay.

Various modifications of the aforesaid embodiment are possible. Someexamples are explained in the following. In the aforesaid embodiment,circumstances around the robot 20 are detected by the sensors 211 to 215corresponding to the five human senses, but a situation detecting unitis not limited to this configuration. Although the aforesaid embodimentuses the humanoid robot 20 capable of autonomous movement by bipedwalking, a robot that acts in response to instructions received from theuser 1 by wireless communication is not limited to the aforesaidconfiguration.

In the aforesaid embodiment, the brain activity detectors 11 detectbrain activity of the user 1, the brain activity analyzing portion 152analyzes the detected brain activity and the outputting portion 154outputs a first action instruction from the user 1 to the robot 20, buta computing unit is not limited to this configuration. In the aforesaidembodiment, the robot controlling portion 341 transmits action signalsto the robot 20 in accordance with action instructions instructed by theoutputting portion 154 (first action instructions/second actioninstructions) and controller 26 controls the robot 20 based on thesecontrol signals, but a robot controlling unit (robot controlling portion341 and controller 26) is not limited to the aforesaid configuration.

In the aforesaid embodiment, a head-mounted display having the display12 as the display unit is used in the user apparatus 10, but the userapparatus can be of any type insofar as it operates based on signalsdetected by the sensors 211 to 215 and other situation detecting unitsin a manner perceivable by the user 1. The aforesaid embodiment isadapted to detect reflex actions of the user 1 using the actiondetectors 18 worn on the arms, legs or other parts of the user's body,but an action detecting unit is not limited to the aforesaidconfiguration. For example, predetermined actions of the user 1 can bedetected using cameras. Detection of fingertip reflex movements isanother option.

The aforesaid embodiment is adapted to output a second actioninstruction instead of a first action instruction in case of apredetermined action of the user 1 being detected by the actiondetectors 18 when pictures acquired by the robot 20 are being displayedon the display 12 of the user apparatus 10, i.e., during operation ofthe user apparatus 10. However, the fact of the user apparatus being inoperation as a precondition for outputting such a second actioninstruction is not limited to the case of pictures being displayed onthe display 12. In other words, the precondition is met no matter howthe user apparatus is in operation insofar as the equipment is operatingin a condition that enables the user to tell whether the robotencountered a sudden unexpected situation.

In the aforesaid embodiment, some functions of the controllers 15 and 26of the user apparatus 10 and robot 20 can be assumed by the managementserver 30, and some functions of the management server 30 can be assumedby the controllers 15 and 26. In the aforesaid embodiment, themanagement server 30 exchanges signals with the user apparatus 10 andthe robot 20. In other words, the user apparatus 10 and the robot 20 areadapted to communicate with each other through the management server 30.However, the user apparatus 10 and the robot 20 can be adapted tocommunicate directly rather than through the management server 30. Insuch case, functions of the management server 30 are preferably assumedby the controllers 15 and 26 of the user apparatus 10 and robot 20.

Although the robot 20 is rented from the shop 3 in the aforesaidembodiment, the present invention can be identically configured using arobot owned by the user 1 and usually kept at home. Optionally, therobot 20 can be made to engage in activities alone rather thanparticipate in family activities. Also possible is to enable themanagement server 30 to communicate with a terminal of the shop 3 sothat the management server 30 can be used, for example, to reserverobots and pay rental fees.

The above embodiment can be combined as desired with one or more of theabove modifications. The modifications can also be combined with oneanother.

According to the present invention, the robot can take appropriatereflex-like action even when the robot encountered a sudden unexpectedsituation.

Above, while the present invention has been described with reference tothe preferred embodiments thereof, it will be understood, by thoseskilled in the art, that various changes and modifications may be madethereto without departing from the scope of the appended claims.

What is claimed is:
 1. A robot control system, comprising: a mobilerobot having a situation detecting unit configured to detect surroundingsituations of the robot; a user apparatus configured to operate in amanner recognizable by a user based on signals detected by the situationdetecting unit; an action detecting unit configured to detect apredetermined action of the user; a brain activity detecting unitconfigured to detect a brain activity of the user; and a robotcontrolling unit configured to control the robot, wherein the userapparatus has a computing unit configured to analyze the brain activityof the user detected by the brain activity detecting unit to output afirst action instruction from the user to the robot, wherein the robotcontrolling unit further configured to control the robot in accordancewith the first action instruction output by the computing unit andconfigured to control, when the predetermined action is detected by theaction detecting unit during operation of the user apparatus, the robotin accordance with a second action instruction different from the firstaction instruction output by the computing unit.
 2. The robot controlsystem according to claim 1, wherein the robot controlling unit isfurther configured to control, when the predetermined action is detectedby the action detecting unit, the robot in accordance with thepredetermined action detected by the action detecting unit.
 3. The robotcontrol system according to claim 1, wherein the situation detectingunit has a camera, wherein the user apparatus has a display configuredto display images acquired by the camera.
 4. The robot control systemaccording to claim 1, wherein the action detecting unit is furtherconfigured to detect a magnitude of motion of a predetermined part of abody of the user, wherein the robot controlling unit is furtherconfigured to control the robot in accordance with the second actioninstruction when the magnitude of motion of the predetermined partdetected by the action detecting unit is equal to or greater than apredetermined value.
 5. The robot control system according to claim 1,wherein the action detecting unit is further configured to detect anacceleration of a predetermined part of a body of the user, wherein therobot controlling unit is further configured to control the robot inaccordance with the second action instruction when the acceleration ofthe predetermined part detected by the action detecting unit is equal toor greater than a predetermined value.
 6. The robot control systemaccording to claim 1, further comprising: a management servercommunicably interconnected with each of the robot and the userapparatus, wherein the management server has the robot controlling unit.7. A robot control method for controlling a mobile robot having asituation detecting unit configured to detect surrounding situations ofthe robot, comprising: detecting a predetermined action of a user;detecting and analyzing a brain activity of the user to output a firstaction instruction from the user to the robot; and controlling the robotin accordance with the first action instruction, wherein the controllingthe robot includes controlling the robot in accordance with a secondaction instruction different from the first action instruction when thepredetermined action is detected while a user apparatus operates in amanner recognizable by the user based on signals detected by thesituation detecting unit.
 8. A user apparatus for a robot control systemconfigured to operate in a manner recognizable by a user based onsignals sent from an external robot and indicating surroundingsituations of the robot, comprising: a communicating unit configured toreceive the signals indicating the surrounding situations of the robotand configured to send a control signal to the robot; an actiondetecting unit configured to detect a predetermined action of the user;a brain activity detecting unit configured to detect a brain activity ofthe user; and a computing unit configured to analyze the brain activityof the user detected by the brain activity detecting unit to output afirst action instruction from the user to the robot, wherein thecomputing unit further configured to output a second action instructiondifferent from the first action instruction output by the computing unitto the communicating unit as the control signal, when the predeterminedaction is detected by the action detecting unit.
 9. The user apparatusfor a robot control system according to claim 8, further comprising: adisplay configured to display images indicating the surroundingsituations of the robot.
 10. The user apparatus for a robot controlsystem according to claim 8, wherein the action detecting unit isfurther configured to detect a magnitude of motion of a predeterminedpart of a body of the user, wherein the computing unit is furtherconfigured to output the second action instruction to the communicatingunit when the magnitude of motion of the predetermined part detected bythe action detecting unit is equal to or greater than a predeterminedvalue.
 11. The user apparatus for a robot control system according toclaim 8, wherein the action detecting unit is further configured todetect an acceleration of a predetermined part of a body of the user,wherein the computing unit is further configured to output the secondaction instruction to the communicating unit when the acceleration ofthe predetermined part detected by the action detecting unit is equal toor greater than a predetermined value.